Markov Logic: A Unifying Framework for Statistical Relational Learning

Interest in statistical relational learning (SRL) has grown rapidly in recent years. Several key SRL tasks have been identified, and a large number of approaches have been proposed. Increasingly, a unifying framework is needed to facilitate transfer of knowledge across tasks and approaches, to compare approaches, and to help bring structure to the field. We propose Markov logic as such a framework. Syntactically, Markov logic is indistinguishable from first-order logic, except that each formula has a weight attached. Semantically, a set of Markov logic formulas represents a probability distribution over possible worlds, in the form of a log-linear model with one feature per grounding of a formula in the set, with the corresponding weight. We show how approaches like probabilistic relational models, knowledge-based model construction and stochastic logic programs are special cases of Markov logic. We also show how tasks like collective classification, link prediction, link-based clustering, social network modeling, and object identification can be concisely formulated in Markov logic. Finally, we briefly describe learning and inference algorithms for Markov logic, and report positive results on a link prediction task. 1. The Need for a Unifying Framework Many (if not most) real-world application domains are characterized by the presence of both uncertainty and complex relational structure. Statistical learning focuses on the former, and relational learning on the latter. Statistical relational learning (SRL) seeks to combine the power of both. Research in SRL has expanded rapidly in recent years, both because of the need for it in applications, and because statistical and relational learning have individually matured to the point where combining them is a feasible research enterprise. A number of key SRL tasks have been identified, including collective classification, link prediction, link-based clustering, social network modeling, object identification, and others. A large and growing number of SRL approaches have been proposed, including knowledge-based model construction (Wellman et al., 1992; Ngo & Haddawy, 1997; Kersting & De Raedt, 2001), stochastic logic programs (Muggleton, 1996; Cussens, 1999), PRISM (Sato & Kameya, 1997), probabilistic relational models (Friedman et al., 1999), relational Markov models (Anderson et al., 2002), relational Markov networks (Taskar et al., 2002), relational dependency networks (Neville & Jensen, 2003), structural logistic regression (Popescul & Ungar, 2003), relational generation functions (Cumby & Roth, 2003), CLP(BN) (Costa et al., 2003), and others. While the variety of problems and approaches in the field is valuable, it makes it difficult for researchers, students and practitioners to identify, learn and apply the essentials. In particular, for the most part, the relationships between different approaches and their relative strengths and weaknesses remain poorly understood, and innovations in one task or application do not easily transfer to others, slowing down progress. There is thus an increasingly pressing need for a unifying framework, a common language for describing and relating the different tasks and approaches. To be most useful, such a framework should satisfy the following desiderata: 1. The framework must subsume both first-order logic and probabilistic graphical models. Otherwise some current or future SRL approaches will fall outside its scope. 2. SRL problems should be representable clearly and simply in the framework. 3. The framework must facilitate the incorporation of domain knowledge into SRL. Because the search space for SRL algorithms is very large even by AI standards, domain knowledge is critical to success. Conversely, the ability to incorporate rich domain knowledge is one of the most attractive features of SRL. 4. The framework should facilitate the extension to SRL of techniques from statistical learning, inductive logic programming, probabilistic inference and logical inference. This will speed progress in SRL by taking advantage of the large extant literature in these areas. In the next section we propose a framework that we believe meets all of these desiderata. We then describe how several SRL approaches and tasks can be formulated in this framework. Finally, we illustrate how existing learning and inference techniques can be applied within it to yield practical algorithms.